Systems of the type outlined in the previous paragraph have been compared to neural networks in human beings; and there has been considerable work in designing electrical circuits which simulate neural networks. The standard approach to a neural network is to propose a learning rule, usually based on a synaptic analogy, with a "synapse" being the point at which a nervous impulse passes from one neuron to another. An important article discussing such systems was written by J. J. Hopfield, and is entitled "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proceedings of the National Academy of Sciences, 79:2554 2558. This article and an accompanying introduction also appear at pages 457 through 464 of "Neurocomputing; Foundations of Research" edited by James A. Anderson and Edward Rosenfeld, MIT Press, Cambridge, Mass, 1988.
One particular neural network circuit implementation model is disclosed in an article entitled "A Learning Algorithm for Boltzmann Machines" by David H. Ackley et al, Cognitive Science, Vol. 9, pp. 147-169, 1985. A Boltzman machine may be considered to be a layered system constructed of units which can be in on or off states, of link weights, connecting "hidden" or associative units with input and output units, and which can take on values from the real numbers. A weight modifies the output of the unit feeding it, and passes the result as an input to the unit which it feeds. In addition to input and output units are "hidden" or "associative" units which assist in processing. It has been proposed heretofore to implement "hidden" or associative units by electrical or electronic circuits. The Boltzmann Machine operates by presenting examples of Input/Output (I/O) patterns to the I/O units, and then allowing the "hidden" or associative units to adjust stochastically to minimize a function referred to as energy. The weights are adjusted so that the hidden or associative units behave in the same way whether the outputs are fixed or left free. The weight adjustment (learning) procedure is local but is computationally expensive and slow, at least in part as a result of the need to implement the "hidden" or associative links by electronic circuits.
Accordingly, a principal object of the present invention is to provide an inexpensive, rapid associative memory and/or processor having a massive information storage capacity.